Calibrating 3d motion capture system for skeletal alignment using x-ray data

ABSTRACT

A processing device receives, from a three-dimensional (3D) motion capture system, initial data representing an initial orientation of a subject user&#39;s body in an initial position. The processing device further receives x-ray data representing at least the portion of the subject user&#39;s body in the initial position. The processing device determines an actual orientation of at least one bone or joint from the portion of the subject user&#39;s body in the initial position as represented in the x-ray data and calibrates the initial orientation of the 3D motion capture system to reflect the actual orientation of the at least one bone or joint in the initial position.

RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/931,602, filed Nov. 6, 2019, the entire contents ofwhich are hereby incorporated by reference herein.

TECHNICAL FIELD

The present disclosure is generally related to computer systems, and ismore specifically related to calibrating a 3D motion capture system forskeletal alignment using x-ray data.

BACKGROUND

Two dimensional (2D) imaging is widely used by doctors and other healthprofessionals to analyze human motion in sports and health applicationsbecause 2D imaging is relatively simple, inexpensive and widelyavailable. Three dimensional (3D) motion visualization is much moreadvanced and provides data, multiple viewing angles, and digital dataanalysis that 2D imaging cannot provide. 3D systems can provide usefulinformation of angles, speed, orientation, etc. which can be used toidentify poor movement for performance or health. 3D motionvisualization, however, requires sensors or markers and technology thatmay take longer to set up and is more expensive.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example, and not by wayof limitation, and can be more fully understood with reference to thefollowing detailed description when considered in connection with thefigures in which:

FIG. 1 depicts a high-level component diagram of an illustrative systemarchitecture, in accordance with one or more aspects of the presentdisclosure.

FIG. 2 depicts a visualization of the subject user in one possibleinitial position, in accordance with one or more aspects of the presentdisclosure.

FIG. 3 depicts one example of an interface for calibrating a 3D motioncapture system for skeletal alignment using x-ray data in accordancewith one or more aspects of the present disclosure.

FIG. 4 is a flow diagram illustrating a method of calibrating a 3Dmotion capture system for skeletal alignment using x-ray data inaccordance with one or more aspects of the present disclosure.

FIG. 5 is a flow diagram illustrating method of determining the actualorientation of at least one bone or joint in an initial position usingx-ray data in accordance with one or more aspects of the presentdisclosure.

FIG. 6 depicts an example computer system which can perform any one ormore of the methods described herein, in accordance with one or moreaspects of the present disclosure.

DETAILED DESCRIPTION

Embodiments for calibrating a 3D motion capture system for skeletalalignment using x-ray data are described. The technology describedherein can be used to derive precise skeletal alignment and movementdata using a combination of x-ray data and a 3D motion capture system.Three-dimensional (3D) motion capture systems derive their informationfrom markers, cameras, or sensors placed on the outside of the subjectuser's body. A calibration method is used to create a reference framefor the movement of the body, so that the relative movement of thesensors and body can be tracked and analyzed. Certain implementationsattempt to align the sensors relative to specific portions of thesubject user's body (e.g., the bones of the user's skeleton). Sincepeople have flesh, muscles, etc. in between the skeleton and the sensorsaffixed to the skin, it can be difficult to know the exact alignment ofthe sensors relative to the skeleton itself and, therefore, difficult tomeasure the precise angles of movement of the skeleton using thesensors. Often times an assumption is made about that orientation of theskeleton inside the body in order to generate the reference frame. Whilethis is adequate for certain implementations, other implementationsrequire a more precise alignment of the motion capture sensors and theskeleton. Without this precise alignment, the reference frame can bemiscalculated the 3D motion capture analysis can be inaccurate and/ormisleading.

Of course the technology does exist to see inside the subject user'sbody. Two-dimensional (2D) radiographs (i.e., x-ray images) or acomputerized tomography scan can capture a user's bones or joints incertain static positions (e.g., supine, standing, and/or sitting). Thesestatic images are less than ideal for motion capture analysis due totheir imprecise nature, their ability to capture only a brief moment intime which can be quite variable, and their limited reproducibility andconsistency possibly leading to measurement error. The x-ray imagesgenerally only capture a single plane (e.g., the sagittal plane, thecoronal plane) and do not offer a 3D perspective. In addition, thestatic images cannot be used to detect rotational (e.g., axial ortransverse plane) or coupled orientation changes.

The implementations described herein address the above and otherconsiderations by calibrating a 3D motion capture system for skeletalalignment using x-ray data. By capturing x-ray data to get preciseskeletal alignment and then combining this x-ray data with 3D motioncapture data to track movement of the subject user's body, the systemdescribed herein can obtain precise skeletal angles and positions duringthe movement. In one embodiment, the subject user stands in a known pose(i.e., a calibration pose) and one or more x-ray images are taken fromone or more angles. For example, an x-ray can be taken from the front ofthe subject user and then from one or more of the other sides. Thisx-ray image(s) can then be used to determine the position (i.e., angles)of certain body parts (e.g., portions of the user's skeleton) in thatknown pose. For example, if an x-ray is taken from the front and theside, the resulting image will illustrate the tilt, obliquity, androtation of the pelvis or other bone or body part in that known pose. Inone embodiment, the system can receive user input (e.g., from atechnician, doctor, or other user) identifying the portions of thesubject user's skeleton that are of interest and can determine one ormore corresponding measurements (e.g., the angles of orientation of theskeleton relative to some global reference frame, or relative to someother element, such as a 3D motion capture sensor).

As described above, conventional motion capture systems either assumethe angles of the skeleton are zero (or some other default value) in thecalibration pose or use the measurements from placed on the outside ofthe body which assume the skeleton is similarly aligned. In embodimentsof the present disclosure, however, x-ray data showing the trueorientation of the skeleton is received as an input to the 3D motioncapture system, and can be used to offset the position of the skeleton.For example, at least one calibration pose is an attention pose whereeach bone is considered to be in a neutral position (i.e., zero degreesof rotation on all three axes). In this pose, the joints are alsoconsidered neutral (i.e., zero degrees of bending). By using the actualskeletal alignment data obtained from the x-ray images and offsettingthe skeletal data in the same reference pose captured by the 3D motioncapture system, the 3D motion capture system now has a much moreaccurate skeletal reference frame to capture precise skeletal anglesthroughout movement of the subject user's body. The input data from thex-ray images is incorporated into the 3D motion capture system to definethe starting skeletal alignment before any calculations are made. Thedata from the x-ray images of the subject user in the reference pose canbe saved into the 3D motion capture system and all motion capture dataand results can be calculated based on movement relative to thereference pose.

This adjustment can be very important for certain applications. Forexample, if the subject user were to perform a simple motion, such as adeep squat (i.e., starting in a standing position and bending the kneesand waist in a deep squat), most sensor based systems must assume thepelvis initially had zero degrees of rotation, tilt, and obliquity atthe starting position. During movement and at the bottom of the deepsquat, the system can produce relative angles of the pelvis, but only inrelation to the assumed starting angles of the pelvis. The x-ray datainput to the system however, might show that the subject user actuallyhas 10 degrees of anterior tilt and 3 degrees of left side obliquity,for example, in the starting position. If this case, the positional dataof the pelvis could be quite misleading at the bottom of the deep squatbecause the initial reference orientation was not correct.

In one embodiment, one feature of the system is to analyze the subjectuser's movement based on the position of the pelvis at the bottom of thedeep squat. With an incorrect reference orientation, the 3D motioncapture system would be providing inaccurate results. Since thepositions and movement of the pelvis, femur, and hips, for example, moveon all three planes simultaneously, the system cannot simply offset theresult data by the initial differences of position provided by the x-raydata and the assumptions made. However, by combining the actual knownpositions of the skeleton provided by the x-ray data, inputting thosepositions into the 3D motion capture system before any movement is made,and then performing 3D calculations of the motion data with the actualpositions of the skeleton, the system can more accurately determine theangles and positions of the skeleton during all parts of the movement,leading to much more precise and useful analysis.

FIG. 1 depicts a high-level component diagram of an illustrative systemarchitecture 100, in accordance with one or more aspects of the presentdisclosure. System architecture 100 includes a computing device 110 anda repository 120 connected to a network 130. Network 130 may be a publicnetwork (e.g., the Internet), a private network (e.g., a local areanetwork (LAN) or wide area network (WAN)), or a combination thereof.

The computing device 110 may be configured to perform dynamic 3D motioncapture for body motion analysis, skeletal alignment, and/or otheranalyses. In one embodiment, computing device 110 may be a desktopcomputer, a laptop computer, a smartphone, a tablet computer, a server,or any suitable computing device capable of performing the techniquesdescribed herein. In one embodiment, computing device 110 receives 3Dmotion capture data 133 from a 3D motion capture system, which can beimplemented in a variety of ways. In one embodiment, a plurality ofmotion capture sensors 142, which may be affixed to one or more bodyparts of a subject user 140 while they are performing a physicalmovement, capture 3D motion capture data 144 corresponding to thesubject user 140. Depending on the implementation, the motion capturesensors 142 can be attached externally to the skin of the subject user140 or internally to the bones of the subject user 140. In otherembodiments, the motion capture sensors 142 may be affixed to anyrelevant object being manipulated by the subject user 140 whileperforming the physical movement, such as to a golf club, baseball bat,tennis racquet, crutches, prosthetics, etc. The 3D motion capture data144 may be received by the computing device 110. In one embodiment, the3D motion capture system is integrated within computing device 110, andso the 3D motion capture data 144 is received internally.

When the 3D motion capture system is external to computing device 110,the 3D motion capture data 144 may be received in any suitable manner.For example, the motion capture sensors 142 may be wireless inertialsensors, each including for example, a gyroscope, magnetometer,accelerometer, and/or other components to measure sensor data includingrelative positional data, rotational data, and acceleration data. The 3Dmotion capture data 144 may include this sensor data and/or other dataderived or calculated from the sensor data. The motion capture sensors142 may transmit the 3D motion capture data 144 including, raw sensordata, filtered sensor data, or calculated sensor data, wirelessly tocomputing device 110 using internal radios or other communicationmechanisms. In other embodiments, the 3D motion capture system may notutilize motion capture sensors 142 and other systems may be used tocapture 3D motion capture data 144, such as an optical system, using oneor more cameras including a marker-based camera system or a marker-lesscamera system, a mechanical motion system, an electro-magnetic system,an infra-red system, etc. In addition, in other embodiments, the 3Dmotion capture data 144 may have been previously captured and stored ina database or other data store. In this embodiment, computing device 110may receive the 3D motion capture data 144 from another computing deviceor storage device where the 3D motion capture data 144 is maintained. Instill other embodiments, the 3D motion capture data 144 may beassociated with other users besides or in addition to subject user 140performing a physical activity.

In one embodiment, motion capture sensors 142 capture the 3D motioncapture data 144 while the subject user 140 is in an initial position(i.e., a calibration position or pose). While in the initial position,the subject user's body, or at least a portion of the body, can remainstill or nearly still, so that the 3D motion capture data 144 caninclude initial data representing an initial orientation of the motioncapture sensors 142. FIG. 2 depicts a visualization 200 of subject user140 in one possible initial position, in accordance with one or moreaspects of the present disclosure. The visualization 200 illustrates the“attention” position, which can be used to calibrate the motion capturesensors 142 affixed to the body of subject user 140. In the attentionposition, subject user 140 can be standing with arms hanging straightdown with palms facing the thighs. The feet can be flat on the ground,parallel, and straight below the hips. Elbows and knees can be locked,with wrists flat, and all fingers pointing straight down. In otherembodiments, other initial positions can be used.

In one embodiment, the motion capture sensors 142 are placed on thesubject user's body, in an attempt to align with at least one bone orjoint while subject user 140 is in the initial position. For example,one or more motion capture sensors 142 can be placed on the hips and/orback of subject user 140 in approximate alignment with the pelvis. Dueto the intervening layers of skin, muscle, or other body tissue, theremay be a difference between the alignment of the motion capture sensors142 and the alignment of the pelvis of subject user 140. Thus, theinitial data representing the initial orientation of the motion capturesensors 142 may not accurately represent the orientation of the pelviswhen the subject user is in the initial position. For example, theinitial data includes first values of tilt and rotation measured by themotion capture sensors 142. In one embodiment, in the absence of anyexternal reference frame, these first values representing the initialorientation of motion capture sensors 142 include zero degrees of tiltand rotation on all three axes (e.g., 0 degrees forward tilt, 0 degreesside tilt, 0 degrees rotation). As noted, these first values may notaccurately represent the actual orientation of the pelvis, or other boneor joint, of subject user 140.

In one embodiment, motion capture sensors 142 further capture the 3Dmotion capture data 144 while the subject user 140 is performing aphysical activity or a physical movement. The physical activity can befor example, swinging a golf club, throwing a ball, running, walking,jumping, sitting, standing, or any other physical activity. Whenperforming the physical activity, the subject user 140 may make one ormore physical body movements that together enable performance of thephysical activity. For example, when performing a physical activity,such as standing form a sitting position, walking, or other activity,the user may rotate their hips and shoulders, swing their arms, hingetheir wrists, turn their pelvis, etc., each of which can be considered aseparate body movement associated with performing the physical activity.Each physical activity may have its own unique set of associated bodymovements. Each physical movement can involve motion of a bone or jointof the subject user 140. Thus, the 3D motion capture data 144 caninclude continuous motion capture data representing dynamic motion of atleast one of a bone or joint of the subject user 140 while they areperforming the physical movement. The continuous nature candifferentiate the 3D motion capture data 144 from a mere static imagecaptured at a single point in time.

In one embodiment, computing device 110 further receives x-ray data 145,such as from an imaging device 130. Imaging device 130 can include forexample, radiography equipment, such as an x-ray generator and detector,computerized tomography equipment, or other imaging equipment. The x-raydata 145 can include a digital data stream representing the x-ray imageof at least a portion of the body of subject user 140, captured byimaging device 143. For example, while subject user 140 is in theinitial position, such as the attention pose described above, imagingdevice 130 can capture the x-ray image, and corresponding x-ray data 145can be sent to computing device 110.

In one embodiment, computing device 110 may include body movementanalysis engine 112. The body movement analysis engine 112 may includeinstructions stored on one or more tangible, machine-readable storagemedia of the computing device 110 and executable by one or moreprocessing devices of the computing device 110. In one embodiment, bodymovement analysis engine 112 receives the 3D motion capture data 144 ofthe subject user 140 performing the physical activity or physicalmovement and receives the x-ray data 145 of subject user 140representing at least a portion of subject user's body while in theinitial position. Body movement analysis engine 112 can use x-ray data145 to refine the motion capture data 344 and improve the readings ofmotion capture sensors 142.

In one embodiment, body movement analysis engine 112 determines anactual orientation of at least one bone or joint from the portion of thesubject user's body in the initial position, as represented in x-raydata 145. In one embodiment, body movement analysis engine 112 generatesan image (i.e., an x-ray image) from the received x-ray data 145, wherethe image is of a portion of the body (e.g., the pelvis) of subject user140. Body movement analysis engine 112 can cause display of the image inan interface on a display device, such as display device 114. Dependingon the embodiment, the display device can be integrated within computingdevice 110, or can be a display device of some other device connected tonetwork 130, or wirelessly connected directly to computing device 110 orany other computing device. FIG. 3 depicts one example of an interface300 for calibrating a 3D motion capture system for skeletal alignmentusing x-ray data in accordance with one or more aspects of the presentdisclosure. In one embodiment, interface 300 is a graphical userinterface including an image display area 310 and a user input area 320.In one embodiment, upon presenting the x-ray image in display area 310of interface 300, body movement analysis engine 112 further receivesinput, such as user input data 146 via input area 320 of interface 300.A surgeon, doctor, technician, health professional, or other user, canprovide user input data 146 including an indication of at least one boneor joint depicted in the x-ray image. As illustrated in FIG. 3, the userinput can include an alignment indicator indicating the at least onebone or joint. In this case, alignment indicator 312 indicates thealignment of the pelvis shown in the x-ray image, and alignmentindicator 314 indicates the alignment of the femur shown in the x-rayimage.

Based on the alignment indicator(s), body movement analysis engine 112can calculate an offset or offsets of the at least one bone or jointrelative to a reference frame. For example, body movement analysisengine 112 can determine that alignment indicator 312 indicates acertain degree of pelvic tilt (e.g., −10 degrees) on a given axis, andthat alignment indicator 314 indicates a certain degree of femur flexion(e.g., 1 degree). In one embodiment, body movement analysis engine 112calculates these offsets in relation to a reference frame, such as anEarth-bound reference frame (i.e., 3-axis coordinate system). In otherembodiments, the offsets can be defined using some other referenceframe, such as a functional pelvic plane, or an anterior pelvic plane(APP) reference frame. For example, the offsets can include secondvalues of tilt and rotation relative to the defined axes (i.e., zerodegrees) of the reference frame. In one embodiment, these offsetsrepresent the actual orientation of the at least one bone or joint inthe initial position.

Referring again to FIG. 1, in one embodiment, having determined theactual orientation of the at least one bone or joint in the initialposition, body movement analysis engine 112 can calibrate the initialorientation of the motion capture sensors 142, as indicated in 3D motioncapture data 144, to reflect the actual orientation. In one embodiment,body movement analysis engine 112 can replace the first values of tiltand rotation measured by the motion capture sensors 142 with the secondvalues of tilt and rotation determined from the x-ray data 145. Forexample, as described above, if the first values representing theinitial orientation of motion capture sensors 142 include zero degreesof tilt and rotation on all three axes, body movement analysis enginecan replace those first values with the second values representing theactual orientation of the at least one bone or joint (e.g., −10 degreesof forward pelvic tilt). As such, body movement analysis engine 112 caninterpret movement of the body of subject user 140, as reflected insubsequently received 3D motion capture data 144, relative to the actualorientation in the initial position. Such as if body movement analysisengine 112 were attempting to analyze the amount of pelvic tilt subjectuser 140 achieved during the performance of a body movement (e.g., adeep squat), the amount of pelvic tilt at the bottom of the deep squatcould be compared to the amount of pelvic tilt in the initial position(i.e., at the start of the deep squat). Since the initial orientationhas been calibrated to reflect the actual orientation of the pelvis inthe initial position, the difference between the measurements will be amore accurate representation of the actual amount of pelvic tiltachieved. In one embodiment, 3D motion capture data 144, x-ray data 145,user input data 146, and data representing the actual orientation of theat least one bone or joint as determined by body movement analysisengine can be stored as part of alignment data 122 in repository 120.

The repository 120 is a persistent storage device that is capable ofstoring alignment data 122 and/or other data, as well as data structuresto tag, organize, and index this data. Repository 120 may be hosted byone or more storage devices, such as main memory, magnetic or opticalstorage based disks, tapes or hard drives, NAS, SAN, and so forth.Although depicted as separate from the computing device 110, in animplementation, the repository 120 may be part of the computing device110 or may be directly attached to computing device 110. In someimplementations, repository 120 may be a network-attached file server,while in other embodiments, repository 120 may be some other type ofpersistent storage such as an object-oriented database, a relationaldatabase, and so forth, that may be hosted by a server machine or one ormore different machines coupled to the via the network 130.

FIG. 4 is a flow diagram illustrating a method of calibrating a 3Dmotion capture system for skeletal alignment using x-ray data inaccordance with one or more aspects of the present disclosure. Themethod 400 may be performed by processing logic that comprises hardware(e.g., circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions run on a processor to perform hardwaresimulation), firmware, or a combination thereof. In one embodiment,method 400 may be performed by computing device 110 including bodymovement analysis engine 112, as shown in FIG. 1.

Referring to FIG. 4, at block 405, method 400 receives, from athree-dimensional (3D) motion capture system, such as motion capturesensors 142 affixed to at least a portion of a subject user's body,initial data representing an initial orientation of the plurality of 3Dmotion capture sensors 142 when the subject user's body is in an initialposition (e.g., the calibration position illustrated in FIG. 2). In oneembodiment, the motion capture sensors are 142 calibrated to the body ofthe subject user 140 while the subject user 140 establishes a pose,providing a baseline orientation of the sensors on the respective bodyparts in a known orientation across the three axes. Computing device 110sends a signal to the sensors to begin the recording. The subject user140 assumes the calibration pose so that the sensors can determine theinitial orientation. The sensors then send the data back the computingdevice 110.

At block 410, method 400 receives x-ray data 145 representing at leastthe portion of the subject user's body in the initial position.

At block 415, method 400 determines an actual orientation of at leastone bone or joint from the portion of the subject user's body in theinitial position as represented in the x-ray data 145. Additionaldetails with respect to how the actual orientation is determined areprovided below with respect to FIG. 5.

At block 420, method 400 calibrates the initial orientation of theplurality of 3D motion capture system to reflect the actual orientationof the at least one bone or joint in the initial position. Additionaldetails with respect to how the initial orientation is calibrated areprovided below with respect to FIG. 5.

At block 425, method 400 receives, from the 3D motion capture system,motion capture data 144 associated with a movement of the subject user'sbody from the initial position to a subsequent position (e.g., a deepsquat). In one embodiment, the motion capture sensors 142 are wirelessinertial sensors, each including a gyroscope, magnetometer,accelerometer, and/or other components to measure relative positionaldata, rotational data, acceleration data, and/or other data. The 3Dmotion capture data 144 includes data representing dynamic motion of atleast one of a bone or a joint of the subject user 140 associated withperforming the physical movement. In one embodiment, computing device110 receives the 3D motion capture data 144 from the motion capturesensors 142 over a wireless communication link (e.g., Bluetooth). Inother embodiments, the 3D motion capture data 144 may have beenpreviously captured and stored in a database or other data store, suchas repository 120. In one embodiment, the 3D motion capture data 144 isaccompanied by a request or instruction to perform a movement analysisfor the subject user 140. The request may be received from a user ofcomputing device 110, from a user of a client device coupled tocomputing device 110 via network 130, or from some other requestor. Inone embodiment, body movement analysis engine 112 receives the 3D motioncapture data 144 and stores the 3D motion capture data 144 in repository120.

At block 430, method 400 analyzes the motion capture data 144 associatedwith the subsequent position in relation to the actual orientation ofthe at least one bone or joint in the initial position.

FIG. 5 is a flow diagram illustrating method of determining the actualorientation of at least one bone or joint in an initial position usingx-ray data in accordance with one or more aspects of the presentdisclosure. The method 500 may be performed by processing logic thatcomprises hardware (e.g., circuitry, dedicated logic, programmablelogic, microcode, etc.), software (e.g., instructions run on a processorto perform hardware simulation), firmware, or a combination thereof. Inone embodiment, method 500 may be performed by computing device 110including body movement analysis engine 112, as shown in FIG. 1.

Referring to FIG. 5, at block 505, method 500 receives x-ray data 145representing at least the portion of the subject user's body in theinitial position, as described above.

At block 510, method 500 generates, from the x-ray data 145, an image ofthe portion of the subject user's body in the initial position.

At block 515, method 500 causes display of the image in an interface(e.g., interface 300 of FIG. 3) on a display device 114.

At block 520, method 500 receives input via the interface 300, the inputcomprising an indication of the at least one bone or joint. Bodymovement analysis engine 112 can receive input, such as user input data146, from a surgeon, or other health professional, including anindication of the bone or joint in the x-ray image. In one embodiment,the interface 300 includes a number of controls through which the usercan provide input data 146 to indicate the bone or joint. For example,the controls can provide the ability to draw a line, such as 312 or 314indicating the alignment of a certain bone in the x-ray image. Asurgeon, radiologist, or other user can use known landmarks visible inthe x-ray to position the lines. For example on the pelvis, a trainedprofessional will know to position one end of the line on a specificpart of the bone and the other on another specific part of the bone.

At block 525, method 500 determines an offset of the at least one boneor joint relative to a reference frame, wherein the offset representsthe actual orientation of the at least one bone or joint in the initialposition. In one embodiment, body movement analysis engine 112 can runalgorithms that take the raw sensor data and compute human readablemotion analysis, for example, taking quaternion sensor data andcomputing Euler angles relative to the three axis of rotation of bonesegments. This can then be converted into joint movement data such asinternal/external rotation, abduction/adduction and flexion/extension ofa joint (e.g., hip, knee, shoulder, etc.), or bony segment (e.g., femur,tibia/fibula, humerus, radius ulna, vertebra, etc.), as well as jointand skeletal contact stresses and joint reaction forces. Furthermore,the bone and joint movement data can take the x-ray data 145 to make anoff-set adjustment of the initial orientation. For example, the initialorientation may assume that a certain bone or other body part has zeroforward or backward bend. The x-ray data 145 may provide initial forwardand backward angle bends for the body parts, which can be an inputparameter to provide the actual orientation of the body part from whichthe relative movement data of the sensors can be offset. In oneembodiment, body movement analysis engine 112 can compute the angle ofthe lines drawn on the x-ray image to determine the initial offset. Thealgorithm will compute the segment and joint movements frame by framecaptured by the sensors and map out the data in graph form. Thisinformation is readily accessible on the computing device right afterthe capture without human intervention or adjustment. The 3D motioncapture sensors 142 can capture continuous movement data across multipleplanes, which offers a substantial improvement over static 2D images.Conventional systems capture the body part in two positions (e.g.,standing and sitting), and uses statistical models to try to extrapolatethe movement, range of motion, and possible impingement. Using thetechniques described herein, body movement analysis engine 112 can getactual movement, range of motion, and impingement angles from actualdata without having to rely on statistical projections.

At block 530, to calibrate the initial orientation of the plurality of3D motion capture sensors 142 to reflect the actual orientation of theat least one bone or joint in the initial position, method 500 replacesfirst values of tilt and rotation measured by the plurality of 3D motioncapture sensors 142 with second values of tilt and rotation determinedfrom the x-ray data 145.

FIG. 6 depicts an example computer system 600 which can perform any oneor more of the methods described herein, in accordance with one or moreaspects of the present disclosure. In one example, computer system 600may correspond to a computing device, such as computing device 110,capable of executing body movement analysis engine 112 of FIG. 1. Thecomputer system 600 may be connected (e.g., networked) to other computersystems in a LAN, an intranet, an extranet, or the Internet. Thecomputer system 600 may operate in the capacity of a server in aclient-server network environment. The computer system 600 may be apersonal computer (PC), a tablet computer, a set-top box (STB), apersonal Digital Assistant (PDA), a mobile phone, a camera, a videocamera, or any device capable of executing a set of instructions(sequential or otherwise) that specify actions to be taken by thatdevice. Further, while only a single computer system is illustrated, theterm “computer” shall also be taken to include any collection ofcomputers that individually or jointly execute a set (or multiple sets)of instructions to perform any one or more of the methods discussedherein.

The exemplary computer system 600 includes a processing device 602, amain memory 604 (e.g., read-only memory (ROM), flash memory, dynamicrandom access memory (DRAM) such as synchronous DRAM (SDRAM)), a staticmemory 606 (e.g., flash memory, static random access memory (SRAM)), anda data storage device 618, which communicate with each other via a bus630.

Processing device 602 represents one or more general-purpose processingdevices such as a microprocessor, central processing unit, or the like.More particularly, the processing device 602 may be a complexinstruction set computing (CISC) microprocessor, reduced instruction setcomputing (RISC) microprocessor, very long instruction word (VLIW)microprocessor, or a processor implementing other instruction sets orprocessors implementing a combination of instruction sets. Theprocessing device 602 may also be one or more special-purpose processingdevices such as an application specific integrated circuit (ASIC), afield programmable gate array (FPGA), a digital signal processor (DSP),network processor, or the like. The processing device 602 is configuredto execute instructions for performing the operations and stepsdiscussed herein.

The computer system 600 may further include a network interface device608. The computer system 600 also may include a video display unit 610(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), analphanumeric input device 612 (e.g., a keyboard), a cursor controldevice 614 (e.g., a mouse), and a signal generation device 616 (e.g., aspeaker). In one illustrative example, the video display unit 610, thealphanumeric input device 612, and the cursor control device 614 may becombined into a single component or device (e.g., an LCD touch screen).

The data storage device 618 may include a computer-readable medium 628on which the instructions 622 (e.g., implementing body movement analysisengine 112) embodying any one or more of the methodologies or functionsdescribed herein is stored. The instructions 622 may also reside,completely or at least partially, within the main memory 604 and/orwithin the processing device 602 during execution thereof by thecomputer system 600, the main memory 604 and the processing device 602also constituting computer-readable media. The instructions 622 mayfurther be transmitted or received over a network via the networkinterface device 608.

While the computer-readable storage medium 628 is shown in theillustrative examples to be a single medium, the term “computer-readablestorage medium” should be taken to include a single medium or multiplemedia (e.g., a centralized or distributed database, and/or associatedcaches and servers) that store the one or more sets of instructions. Theterm “computer-readable storage medium” shall also be taken to includeany medium that is capable of storing, encoding or carrying a set ofinstructions for execution by the machine and that cause the machine toperform any one or more of the methodologies of the present disclosure.The term “computer-readable storage medium” shall accordingly be takento include, but not be limited to, solid-state memories, optical media,and magnetic media.

Although the operations of the methods herein are shown and described ina particular order, the order of the operations of each method may bealtered so that certain operations may be performed in an inverse orderor so that certain operation may be performed, at least in part,concurrently with other operations. In certain implementations,instructions or sub-operations of distinct operations may be in anintermittent and/or alternating manner.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other implementations will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the disclosure should, therefore, bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

In the above description, numerous details are set forth. It will beapparent, however, to one skilled in the art, that the aspects of thepresent disclosure may be practiced without these specific details. Insome instances, well-known structures and devices are shown in blockdiagram form, rather than in detail, in order to avoid obscuring thepresent disclosure.

Some portions of the detailed descriptions above are presented in termsof algorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise, as apparent from the followingdiscussion, it is appreciated that throughout the description,discussions utilizing terms such as “receiving,” “determining,”“selecting,” “storing,” “setting,” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

The present disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions, each coupled to a computer system bus.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear as set forth in thedescription. In addition, aspects of the present disclosure are notdescribed with reference to any particular programming language. It willbe appreciated that a variety of programming languages may be used toimplement the teachings of the present disclosure as described herein.

Aspects of the present disclosure may be provided as a computer programproduct, or software, that may include a machine-readable medium havingstored thereon instructions, which may be used to program a computersystem (or other electronic devices) to perform a process according tothe present disclosure. A machine-readable medium includes any procedurefor storing or transmitting information in a form readable by a machine(e.g., a computer). For example, a machine-readable (e.g.,computer-readable) medium includes a machine (e.g., a computer) readablestorage medium (e.g., read only memory (“ROM”), random access memory(“RAM”), magnetic disk storage media, optical storage media, flashmemory devices, etc.).

The words “example” or “exemplary” are used herein to mean serving as anexample, instance, or illustration. Any aspect or design describedherein as “example” or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or”. That is, unlessspecified otherwise, or clear from context, “X includes A or B” isintended to mean any of the natural inclusive permutations. That is, ifX includes A; X includes B; or X includes both A and B, then “X includesA or B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Moreover, use of the term “an embodiment” or “one embodiment” or“an implementation” or “one implementation” throughout is not intendedto mean the same embodiment or implementation unless described as such.Furthermore, the terms “first,” “second,” “third,” “fourth,” etc. asused herein are meant as labels to distinguish among different elementsand may not necessarily have an ordinal meaning according to theirnumerical designation.

What is claimed is:
 1. A method comprising: receiving, from athree-dimensional (3D) motion capture system, initial data representingan initial orientation of a subject user's body in an initial position;receiving x-ray data representing at least the portion of the subjectuser's body in the initial position; determining an actual orientationof at least one bone or joint from the portion of the subject user'sbody in the initial position as represented in the x-ray data; andcalibrating the initial orientation of the 3D motion capture system toreflect the actual orientation of the at least one bone or joint in theinitial position.
 2. The method of claim 1, further comprising:generating, from the x-ray data, an image of the portion of the subjectuser's body in the initial position; and causing display of the image inan interface on a display device.
 3. The method of claim 2, whereindetermining the actual orientation of the at least one bone or joint inthe initial position comprises: receiving input via the interface, theinput comprising an indication of the at least one bone or joint; anddetermining an offset of the at least one bone or joint relative to areference frame, wherein the offset represents the actual orientation ofthe at least one bone or joint in the initial position.
 4. The method ofclaim 1, wherein calibrating the initial orientation of the 3D motioncapture system to reflect the actual orientation of the at least onebone or joint in the initial position comprises replacing first valuesof tilt and rotation measured by the 3D motion capture system withsecond values of tilt and rotation determined from the x-ray data. 5.The method of claim 1, wherein receiving the initial data representingthe initial orientation of the subject user's body comprises receivingthe initial data from a plurality of 3D motion capture sensors affixedto at least a portion of the subject user's body.
 6. The method of claim5, further comprising: receiving, from the plurality of 3D motioncapture sensors, motion capture data associated with a movement of thesubject user's body from the initial position to a subsequent position;and analyzing the motion capture data associated with the subsequentposition in relation to the actual orientation of the at least one boneor joint in the initial position.
 7. The method of claim 6, wherein themotion capture data comprises one or more of positional data, rotationaldata, or acceleration data measured by the plurality of 3D motioncapture sensors.
 8. A system comprising: a memory device storinginstructions; a processing device coupled to the memory device, theprocessing device to execute the instructions to perform operationscomprising: receiving, from a three-dimensional (3D) motion capturesystem, initial data representing an initial orientation of a subjectuser's body in an initial position; receiving x-ray data representing atleast the portion of the subject user's body in the initial position;determining an actual orientation of at least one bone or joint from theportion of the subject user's body in the initial position asrepresented in the x-ray data; and calibrating the initial orientationof the 3D motion capture system to reflect the actual orientation of theat least one bone or joint in the initial position.
 9. The system ofclaim 8, wherein the processing device to execute the instructions toperform further operations comprising: generating, from the x-ray data,an image of the portion of the subject user's body in the initialposition; and causing display of the image in an interface on a displaydevice.
 10. The system of claim 9, wherein determining the actualorientation of the at least one bone or joint in the initial positioncomprises: receiving input via the interface, the input comprising anindication of the at least one bone or joint; and determining an offsetof the at least one bone or joint relative to a reference frame, whereinthe offset represents the actual orientation of the at least one bone orjoint in the initial position.
 11. The system of claim 8, whereincalibrating the initial orientation of the 3D motion capture system toreflect the actual orientation of the at least one bone or joint in theinitial position comprises replacing first values of tilt and rotationmeasured by the 3D motion capture system with second values of tilt androtation determined from the x-ray data.
 12. The system of claim 8,wherein receiving the initial data representing the initial orientationof the subject user's body comprises receiving the initial data from aplurality of 3D motion capture sensors affixed to at least a portion ofthe subject user's body.
 13. The system of claim 8, wherein theprocessing device to execute the instructions to perform furtheroperations comprising: receiving, from the plurality of 3D motioncapture sensors, motion capture data associated with a movement of thesubject user's body from the initial position to a subsequent position;and analyzing the motion capture data associated with the subsequentposition in relation to the actual orientation of the at least one boneor joint in the initial position.
 14. The system of claim 13, whereinthe motion capture data comprises one or more of positional data,rotational data, or acceleration data measured by the plurality of 3Dmotion capture sensors.
 15. A non-transitory computer-readable storagemedium storing instructions that, when executed by a processing device,cause the processing device to perform operations comprising: receiving,from a three-dimensional (3D) motion capture system, initial datarepresenting an initial orientation of a subject user's body in aninitial position; receiving x-ray data representing at least the portionof the subject user's body in the initial position; determining anactual orientation of at least one bone or joint from the portion of thesubject user's body in the initial position as represented in the x-raydata; and calibrating the initial orientation of the 3D motion capturesystem to reflect the actual orientation of the at least one bone orjoint in the initial position.
 16. The non-transitory computer-readablestorage medium of claim 15, wherein the instructions cause theprocessing device to perform further operations comprising: generating,from the x-ray data, an image of the portion of the subject user's bodyin the initial position; and causing display of the image in aninterface on a display device.
 17. The non-transitory computer-readablestorage medium of claim 16, wherein determining the actual orientationof the at least one bone or joint in the initial position comprises:receiving input via the interface, the input comprising an indication ofthe at least one bone or joint; and determining an offset of the atleast one bone or joint relative to a reference frame, wherein theoffset represents the actual orientation of the at least one bone orjoint in the initial position.
 18. The non-transitory computer-readablestorage medium of claim 15, wherein calibrating the initial orientationof the 3D motion capture system to reflect the actual orientation of theat least one bone or joint in the initial position comprises replacingfirst values of tilt and rotation measured by the 3D motion capturesystem with second values of tilt and rotation determined from the x-raydata.
 19. The non-transitory computer-readable storage medium of claim15, wherein receiving the initial data representing the initialorientation of the subject user's body comprises receiving the initialdata from a plurality of 3D motion capture sensors affixed to at least aportion of the subject user's body.
 20. The non-transitorycomputer-readable storage medium of claim 19, wherein the instructionscause the processing device to perform further operations comprising:receiving, from the plurality of 3D motion capture sensors, motioncapture data associated with a movement of the subject user's body fromthe initial position to a subsequent position; and analyzing the motioncapture data associated with the subsequent position in relation to theactual orientation of the at least one bone or joint in the initialposition.